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1. Introduction 

Let us consider a unitary irreducible representation (jr, 7i) of a sim- 
ple, non-compact and connected algebraic Lie group G. Let us de- 
note by K a maximal compact subgroup of G. According to Harish- 
Chandra, the Lie algebra submodule Hk of fT-finite vectors of n con- 
sists of analytic vectors for the representation, i.e. for all v G Hk the 
orbit map 

f v :G^H, g i-> n(g)v 

is real analytic. For these functions /„ we determine, and in full gener- 
ality, their natural domain of definition as holomorphic functions (see 
Theorem 5.1 below): 

Theorem 1.1. Let (tt,H) be a unitary irreducible representation ofG. 
Let v G H be a non-zero K-finite vector and f v be the corresponding 
orbit map. Then there exists a unique maximal G x K c -invariant do- 
main D w C Gc, independent of v, to which f v extends holomorphically. 
Explicitly: 

(i) D n = Gc if ii is the trivial representation. 

(ii) D v = E + K c if G is Hermitian and n is a non-trivial highest 
weight representation. 

(iii) D-x = H~ Kc if G is Hermitian and n is a non-trivial lowest 
weight representation. 

(iv) D-X = EKc in all other cases. 

In the theorem above S, S + , S~ are certain G-domains in Xc = 
Gc/Kc over X = G/K with proper G-action. These domains are 
studied in this paper because of their relevance for the theorem above 
(see [5]). Let us mention that S is the familiar crown domain and that 
the inclusion EK C C D n traces back to our joint work with Robert 
Stanton ([6], [7]). 
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2. Notation 

Throughout this paper G shall denote a connected simple non-compact 
Lie group. We denote by G c the universal complexification of G and 
suppose: 

• G C G c ; 

• Gc is simply connected. 

We fix a maximal compact subgroup K < G and form 

X = G/K , 

the associated Riemannian symmetric space of the non-compact type. 
The universal complexification Kq of K will be realized as a subgroup 
of G c . We set 

Xc = Gc/ Kc 

and call Xc the affine complexification of X. Note that 

X ^ X c , c/K i-> #fT c 

defines a G-equivariant embedding which realizes X as a totally real 
form of the Stein symmetric space Xc- We write x = K c G X c for 
the standard base point in Xc- 

However, the natural complexification of X is not Xc, but the crown 
domain E C Xc whose definition we recall now. We shall provide the 
standard definition of S, see [1]. 

Lie algebras of subgroups L < G will be denoted by the correspond- 
ing lower case German letter, i.e. 1 < q; complexifications of Lie alge- 
bras are marked with a C-subscript, i.e. lc is the complexification of 
I. 

Let us denote by p the orthogonal complement to t in q with respect 
to the Cartan-Killing form. We set 

{l = {Y G p I spec(ad Y) C (-tt/2, tt/2)} . 

Then 

E = G exp(iQ) • x c Xc 

is a G-invariant neighborhood of X in Xc, commonly referred to as 
crown domain. Sometimes it is useful to have an alternative, although 
less invariant picture of the crown domain: if a C p is a maximal 
abelian subspace and f2 := ft D p, then 



(2.1) 



S = Gexp(if2) • xq . 
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The set is nicely described through the restricted root system E = 
S( ,a): 

n = {Y e a | a(Y) < ir/2 VaeS}. 

If W is the Weyl group of £, then we note that f2 is W-invariant. 

Sometimes we will employ the root space decomposition g = o © 
m ® ©qgs a with m = 3t(a) as usual. We choose a positive system 
S + CS and form the nilpotent subalgebra n = aeS + Q a - 

2.1. The example of G = S1(2,R) 

For illustration and later use we will exemplify the above notions at 
the basic case of G = Sl(2, R). 

We let K = SO(2, R) be our choice for the maximal compact sub- 
group and identify X = G/K with the upper half plane D + := {z G 
C | Imz > 0}. We recall that 

X c = P X (C) x F\C) \ diagpP^C)] 

with Gc acting diagonally by fractional linear transformations. The 
G-embedding of X = D + into Xc is given by 

z i ^ (z, z) e X c ■ 

If D~ denotes the lower half plane, then the crown domain is given by 

S = D + x D~ C X c ■ 

In addition we record two G-domains in X c which sit above S, 
namely: 

(2.2) S+ = D + x P^C) \ diagpP^C)] , 

(2.3) S" = P X (C) x L>-\diag[P 1 (C)]. 
Observe that S = S + n 

3. Remarks on G-invariant domains in Xc with proper 
action 

One defines elliptic elements in Xc by 

X c , e ii = Gexp(ip) ■ x = Gexp(m) • x . 

The main result of [1] was to show that S is a maximal domain in Xc, e ii 
with G-action proper. In particular, G acts properly on S. 

It was found in [5] that S in general is not a maximal domain in 
X c for proper G-action: the domains S + and E~ from (2.2)-(2.3) yield 
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counterexamples. To know all maximal domains is important for the 
theory of representations [5], Sect. 4. 

That S in general is not maximal for proper action is related to the 
unipotent model for the crown which was described in [5]. To be more 
precise, we showed that there exists a domain A C n containing such 
that 

(3.1) S = G exp(iA) • x . 

Now there is a big difference between the unipotent parametrization 

(3.1) and the elliptic parametrization (2.1): If we enlarge the result is 
no longer open; in particular, X C c n is not a domain. On the other hand, 
if we enlarge the open set A the resulting set is still open; in particular 
ATc,u := Gexp(m) -x is a domain. Thus, if there were a bigger domain 
than S with proper action, then it is likely by enlargement of A. 

We need some facts on the boundary of S. 

3.1. Boundary of S 

Let us denote by dE the topological boundary of S in X c . One shows 
that 

d e i\E := Gexp(idil) ■ x C <9S 

(cf. [7]) and calls <9 e uS the elliptic part of <9S. We define the unipotent 
part d u E of <9H to be the complement to the elliptic part: 

<9 U H = <9S \ d c nE . 

The relevance of <9 U H is as follows. Let X C D C Xc denote a G- 
domain with proper G-action. Then D fl 9 e nS = by the above cited 
result of [1]. Thus if D <f_ S, then one has 

Let us describe <9 U S in more detail. For FGowe define a reductive 
subalgebra of gc by 

Q C [Y] = {ZeQ C \e- 2 ^ d ^oa(Z) = Z} 

with a the Cartan involution on gc which fixes 6 + ip. Then there is a 
partial result on <9 U S, for instance stated in [2]: 

(3.2) d u E C {G exp(e) exp(iF) ■ i | V e <9f2, 

(3.3) ^ e E Q C [Y] n ifl nilpotent} . 

If Y is such that only one root, say a, attains the value n/2, then we 
call Y and as well the elements in the boundary orbit G exp(e) exp(iY) ■ 
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x regular. Accordingly we define the regular unipotentboundary 9 U;reg S 
{z E d u E | z regular}. Note that q c [Y] is of especially simple form for 
regular Y, namely 

g c [Y] = ia © m © Q[a]~ e © iQ[a] e 

where g[a\ = g a © Hence, in the regular situation, one can choose 
e above to be in ijj[a;] e + ia. We summarize our discussion: 

Proposition 3.1. Let X C D C X c be a G -invariant domain with 
proper G -action which is not contained in S. Then D n 9 Uireg S 7^ 0. 
More precisely, there exists Y e <9f2 regular (with a£E £/ie unique root 
attaining n/2 on Y ) and a non-zero nilpotent element e G ig[o;] 6 ' + 
swc/i that 



4. Maximal domains for proper action 

The aim of this section is to classify all maximal G-domains in Xq 
which contain X and maintain proper action. The answer will depend 
whether G is of Hermitian type or not. 

4.1. Non-Hermitian groups. 

The objective is to prove the following theorem: 

Theorem 4.1. Suppose that G is not of Hermitian type. If X C D C 
Xq is a G -invariant domain with proper G -action, then D C S. 

Before we can give the proof of the theorem some preparation is 
needed. The proof relies partly on a structural fact characterizing non- 
Hermitian groups (see Lemma 4.4 below) and on a precise knowledge 
of the basic case of G — Sl(2, R). 

Let us begin with the relevant facts for G = S1(2,R). With E = 



a = R-T and n = R ■ E . 

Note that Q = (-vr/4, vr/4)T. 

The a slight modification of results in [5], Sect. 3 and 4 yield: 

Lemma 4.2. Let G = Sl(2, R) and J C R be an open subset. Then 

Ej := Gexp(iJ ■ E) ■ x 
is a G-invariant open subset of X c and the following holds: 



exp(e) exp(iY) ■ x e d u>reg E n D . 
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(i) G does not act properly if {— 1,1} C J . 

(ii) S = «(_!,!). 

(iii) S+ = S ( _i )00) . 

(iv) S" = S(-oc,l)- 

We also need that dE is a fiber bundle over the affine symmetric 
space G/H where H = SO e (l, 1). Notice that H is the stabilizer of the 
boundary point 

z H := exp(-ivrT/4) • x = (1, -1) G <9 e iiS . 

Write r for the involution on G, resp. q, fixing H, resp. f), and 
denote by q = f) + q the corresponding eigenspace decomposition. The 
f)-module q breaks into two eigenspaces q = q + © q with 

q^ 1 = R • e ± where e ± = 

Finally write 

C = M> • e+ U M> • e~ 
and C x =C\ {0}. Note that both C and C x are //-stable. We cite [5], 
Th. 3.1: 

Lemma 4.3. Let G = S1(2,R). Then the map 

Gx H C^dE, [g,e] ^ gexp(ie) ■ z H 
is a G-equivariant homeomorphism. Moreover, 

(i) cknS = G-z H ~ G/H, 

(ii) d u E = Gexp(iC x ) ■ z H ~G x H C x , 

(iii) <9 U S = Gexp(i_E) • rr II Gexp(— iE 1 ) • xq. 

As a last piece of information we need a structural fact which is only 
valid for non-Hermitian groups. 

Lemma 4.4. Suppose that G is not of Hermitian type. Then for all 
a G £ and E G Q a there exists an m G M = Z K (a) such that 

Ad(m)E = -E. 

Proof. Let us remark first that we may assume that G is of adjoint 
type. If G is complex, then the assertion is clear as T := exp(ia) C M 
provides us with the elements we are looking for. More generally for 
dimg a > 1 one knows (Kostant) that M = exp(m) acts transitively 
on the unit sphere in g a (cf. [4]). 

In the sequel we use the terminology and tables of the classification 
of real simple Lie algebras as found in the monograph [3], App. C. As 
G is not Hermitian, Kostant 's result leaves us with the following cases 
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for q: sl(n, R) for n > 3, so(p, q) for 0, 2 ^ p, q and p + q > 2, E I, 
E II, EV,E VI, E VIII, E IX, F I and G. 

Now we make the following observation. The lemma is true for G = 
Sl(3, M) as a simple matrix computation shows. Suppose that a is 
such that it can be put into an A 2 -subsystem of E. As dimg" is one- 
dimensional (by our reduction) this means that we can put E G g a 
in a subalgebra isomorphic to sl(3, R). Now it is important to recall 
the nature of the component group of M, see [3], Th. 7.55. It follows 
that the M-group of Sl(3, M) (isomorphic to (Z/2Z) 2 ) embeds into the 
M- group of G. 

The A 2 -reduction described above deletes most of the cases in our 
list. We remain with the orthogonal cases so(p, q) for 0, 2 ^ p,q and 
p 7^ q. A simple matrix computation, which we leave to the reader, 
finishes the proof. □ 

Proof, (of Theorem 4.1) Suppose that G is not of Hermitian type. Let 
X C D C S be a G-invariant domain with proper G-action which is 
not contained in S. We shall show that D does not exist. 

According to Proposition 3.1 we find a regular F G dfl and a non- 
zero nilpotent e G 0c IX] H ig such that 

exp(e) exp(iY) ■ x G <9 Ujreg H fl D . 

Let a G E be the root corresponding to F. Write Y = Y a + Y' with 
F a , F'Ga such that a(Y') = 0. It is known that Y a G dQ and F' G 
Hence we may use s[(2)-reduction which in conjunction with Lemma 
4.3 implies the existence of E a G Q a such that: 

• {E a ,9(E a ), [E a ,9(E a )]} is an sl(2)-triple, 

• exp(iE a ) exp(iF') • x G <9 Uircg H fl D, 

Now, as G is not of Hermitian type, Lemma [?] implies that there exists 
an element m G M such that Ad(m)E a = —E a . Hence 

exp(-iE a ) exp(iF') • x G <9 UirC gS 

as well. But this contradicts Lemma 4.2 (i). □ 



4.2. Hermitian groups 

Let now G be of Hermitian type and G C P~K C P + be a Harish- 
Chandra decomposition of G in G C - We define flag varieties 

F + = G C /K C P + and F~ = G c /K c p- 

and inside of them we declare the flag domains 

D + = GK i> K I' and D~ = GK c p-/K c p- . 
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Then 

(4.1) X c ^F+xF", gK c ^(gK c P + ,gK c p-) 

identifies Xq as a Zariski open affine piece of F + x F~ . In more de- 
tail: As G is of Hermitian type, there exist wo G Ng c (Kc) such that 
w P ± w 1 = P T . In turn, this element induces a Gc-equivariant bi- 
holomorphic map: 

: F + -> F~, 3F C P + gw K c P- . 
With that the embedding (4.1) gives the following identification of Ac: 

(4.2) X c = {(*, «;) e F + x F" | <j)(z) j w} , 

where j stands for the transversality notion in the flag variety F~ . We 
recall what it means to be transversal. First note that the notion is 
Gc-invariant, i.e. for z, w G F~ and g G Gc one has z j w if and only 
if gz j gw. Now for the base point z~ = K C P~ G F~ one has z~ j w if 
and only if w G P~w z~ . 

We keep the realization of Xq in F + x F~ (cf. (4.1) in mind and 
recall the description of S: 

E = D + x D- 

(see[7]). 

For subsets X 1 * 1 C F 1 * 1 we write X + x T X for those elements (x + , x ) G 
X + x X~ which are transversal, i.e. 4>(x + ) jx~. With this terminology 
in mind we finally define 

2+ = D + x T F~, 
S - = F + x T £T. 

4.2.1. 5as«c structure theory ofE + and E~ . It is obvious that both S + 
and H~ are open and G-invariant. However, as was pointed out by the 
referee, it is a priori not clear that they are connected. In order to see 
this let p + : S + — > D + be the projection onto the first factor. Likewise 
we define p_ : S~ — > 

Proposition 4.5. Lei e G {— , +}. TTie map p e : S e — > F e induces the 
structure of a holomorphic fiber bundle with fiber isomorphic to P e . 

Proof. We confine ourselves with the case e = +. 

As p + is G-equivariant and D + is G-homogeneous, it is sufficient to 
determine the fiber p^ r 1 (z + ). Recall that z + = K C P + G F + is the base 
point. Now 

p-\z + ) = {{z + ,w) G F + x F~ | <f){z + ) T w} . 
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Observe that <fi(z + ) = w z~ and that w z~ j w is equivalent to z~ j 
w^w. By the definition of transversality this means that w^w G 
P~w z~ ortoG w P~w z~ . It is no loss of generality to assume that 
Wq = Wq 1 . So we arrive at w G P + z~ and this concludes the proof of 
the proposition. □ 

Corollary 4.6. Both S + and 5~ are contractible. 

It was observed by the the referee that Proposition 4.5 allows the 
following interesting reformulation. 

Corollary 4.7. The map 

Gx K P + ^E + , [g,p]^ (gz + ,gpz-) 

is a G-equivariant diffeomorphism. In particular S + is G-biholomorphic 
toT Q)l D + , the antiholomorphic tangent bundle of D + . Likewise, S~ is 
G-biholomorphic to T Q,l D~ . 

Corollary 4.7 combined with the Harish- Chandra decomposition im- 
plies that H e ~ D e x P e as complex manifolds. In particular S e is 
Stein. 

The fact that Kq normalizes P e allows us to speak of G x P e -invariant 
domains in Xq- It follows from (4.1) and Corollary 4.7 that S e is GxP e - 
invariant. 

Proposition 4.8. Let e G { — ,+}. The real group G acts properly 
on E e . Moreover E e is a maximal G x P e -invariant domain in Xc for 
proper G-action. 

Proof. As the G-action is proper on D e , it follows that G acts properly 
on S e . In the sequel we deal with e = + only. It remains to show that 
S + is a maximal G x P + -invariant domain in Xc for proper G-action. 
We argue by contradiction and suppose that D D S + is a G x P + - 
domain in Xc with proper G-action. Then D = (Dq x F~) fl Xc with 
D ^ D + a G-domain with proper action. Now recall the following 
facts: 

• There are only finitely many G-orbits in F + . 

• There are precisely two orbits with proper G-action: D + and 
<t>-\D-). 

The assertion follows. □ 

Remark 4.9. Suppose that G is of Hermitian type. Then it can be 
shown that if X C D C Xc is a G-invariant domain with proper G- 
action, then D CB + or D C 2~ . 

As we will not need this fact, we refrain from a proof. 
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If D C X<c is a subset, then we write DKc for its preimage in G ■ 
under the canonical projection G c — > X c . 

Proposition 4.10. The following assertions hold: 

(i) Z+K c = GK C P+, 

(ii) E-K C = GK C P-. 

Proof. It suffices to prove (i). Recall the embedding (4.1), and the 
definition of transversality condition. We deduce that P + C E + K c . As 
E+K c is G x Kc-invariant, it follows that GP + K C = GK C P + C E+K c . 

Conversely, Corollary 4.7 implies that GP + maps onto S + and thus 
2+ c GP + K C . □ 

We conclude this subsection with some easy facts on the structure 
of S + and S~ which will be used later on. 

4.2.2. Unipotent model for S + and We begin with the unipotent 
parameterization of H + and Some terminology is needed. 

According to C. Moore, E is of type C n or BC n . Hence we find a 
subset {71, . . . , 7„} of long strongly orthogonal restricted roots. We fix 
Ej G g 7j such that {Ej,8(Ej), [Ej,9Ej]} becomes an s[(2)-triple. Set 
Tj := 1/2 [Ej, 6Ej] and note that 

n 

n = 0(-7r/2,7r/2)T i . 
j'=i 

We set V = 0" =1 K • Ej and take a cube inside V by 

A = 0(-1,1)^. 

i=i 

In [5], Sect. 8, we have shown that 

S = Gexp(iA) • xo • 

In this parametrization of S the unipotent boundary piece has a simple 
description: 

(4.3) d u E = Gexp(idA) ■ x . 

The strategy now is to enlarge S by enlarging A while maintaining 
that the object stays a domain on which G acts properly. But now 
we have to be a little bit careful with our choice of Ej. Replacing Ej 
by — Ej has no effect for the matters cited above, but for the sequel. 
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Our choice is such that 71, . . . ,j n are positive roots (this determines 
the non-compact roots in S + uniquely). We set 

n n 

A + = 0(-l,cx))£ , i and A" = ®(-oo, l)E 3 . 

3=1 3=1 

Then, a direct generalization of Lemma 4.2(iii),(iv) yields: 

Proposition 4.11. The following assertions hold: 

(i) E + = Gexp(iA + ) • x , 

(ii) H~ = G exp(iA~) ■ x . 

Remark 4.12. If we define subcones of the nilcone Af C g by 



M + = Ad(K) 



and N = -A/" + , 



0[O,oo)£, 

then one can show that the maps 

G x K Af ± -> E ± , [g,Y] ^gexp(iY) ■ x 
are homeomorphic. 



5. Representation theory 

Let (tt, H) be a unitary representation of G and Tix the underlying 
Harish- Chandra module of ^-finite vectors. Notice that TCk is natu- 
rally a module for K c . 

We say that (tt, Ti) is a highest, resp. lowest, weight representation 
if G is of Hermitian type and p + = Lie(P + ), resp. p - , acts on Hk in 
a finite manner. 

We turn to the main result of this paper. 

Theorem 5.1. Let (tt, Ti) be a unitary irreducible representation ofG. 
Let v eH be a non-zero K-finite vector and 

f v :G^n, g^ n(g)v 

the corresponding orbit map. Then there exists a unique maximal G x 
K c -invariant domain D n C G C; independent of v, to which f v extends 
holomorphically. Explicitly: 

(i) D w = (•' ■■ if n is the trivial representation. 

(ii) D w = S + Kq if G is Hermitian and n is a non-trivial highest 
weight representation. 

(iii) D w = E~Kq if G is Hermitian and ti is a non-trivial lowest 
weight representation. 
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(iv) D n = EK<c in all other cases. 

Proof. If 7r is trivial, then the assertion is clear. So let us assume that 
7r is non-trivial in the sequel. Fix a nonzero X-finite vector v and 
consider the orbit map /„ : G — > H. We recall the following two facts: 

• f v extends to a holomorphic G-equivariant map /„ : EK C — > 7i 
(see [7], Th. 1.1). 

• If D v C Gc is a G x i^ c -invariant domain to which extends 
holomorphically, then G acts properly on D v /Kc (see [5], Th. 
4.3) 

We begin with the case where G is not of Hermitian type. Here 
the assertion follows from the bulleted items above in conjunction with 
Theorem 4.1. 

So we may assume for the remainder that G is of Hermitian type. If 
7r is a highest weight representation, then it is clear that f v extends to a 
holomorphic map GKcP + — > 7i. Thus, in this case 2 + fCc = GK<cP + 
(cf. Proposition 4.10 ) is a maximal domain of definition for f v by 
Proposition 4.8 and the second bulleted item from above. Likewise, if 
(tt,H) is a lowest weight representation, then E~K C is a maximal do- 
main of definition of /„. As both S + and S~ are simply connected with 
sufficiently regular boundary, it follows that these maximal domains are 
in fact unique. 

It remains to show: 

• If /„ extends holomorphically on a domain D D S such that 
-Dn[H + \2] 7^ 0, then (ir, H) is a highest weight representation. 

• If /„ extends holomorphically p on a domain D D S such that 
Dfl[H"\S] ^ 0, then (n,TC) is a lowest weight representation. 

It is sufficient to deal with the first case. So suppose that /„ extends 
to a bigger domain D such that D n [S + \ S] ^ 0. Taking derivatives 
and applying the fact that dn(U(gc))v — 7~Lk, we see that f u extends 
to D for all u G H.k- By Proposition 3.1, (4.3) and our assumption we 
find 1 < j < n be such that exp(iEj) exp(iY) ■ xq e D for some Fg O 
with n fj(Y) = 0. Let Gj < G be the analytic subgroup corresponding to 
the s[(2)-triple {Ej,6(Ej), [Ej,8(Ej)]}. Basic representation theory of 
type I-groups in conjunction with [5], Th. 4.7, yields that rc\ breaks 

into a direct sum of highest weight representations. Applying Nk(o) 
(which in particular permutes the Gk and preserves Tlx) we see that 
above matters hold for any other Gk as well (note that Y might change 
but this does not matter as f2 is A/"^(a)-invariant). It follows that n is a 
highest weight representation and completes the proof of the theorem. 
□ 
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Remark 5.2. The domains E, E + and H~ are independent of the choice 
of the connected group G. Accordingly, the above theorem holds for 
all simple connected non-compact Lie groups G, i.e. we can drop the 
assumption that G C Gq and Gc simply connected. 

Problem 5.3. The above theorem should hold true for all irreducible 
admissible Banach representations of G under the reservation that (i) 
gets modified to : D w = Gc if it is finite dimensional. 
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